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(54) Colour image processing method 



(57) Input and output units using different signal 
processing systems cannot be connected to a single im- 
age input apparatus. On the basis of the signal format 



of image data acquired by the input unit and the signal 
format of image data that can be processed by the out- 
put unit, the signal format of image data to be supplied 
from the input unit to the output unit is controlled. 
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Description 

FIELD OF THE INVENTION 

[0001] The present invention relates to an image 
processing apparatus and method thereof and, more 
particularly, to image processing based on the relation- 
ship between an input system and an output system of 
an image signal. 

BACKGROUND OF THE INVENTION 

[0002] Along with the recent development of image 
processing technology, image input apparatuses such 
as a digital camera for sensing an object and outputting 
its image signal are showing a remarkable consumma- 
tion rush. 

[0003] Image input apparatuses are roughly classified 
into a type that processes RGB signals as colorimetric 
data of an object and a type that executes multiple spec- 
tral distribution data processing by estimating the color- 
imetric data of an object illuminated with ambient light 
using the spectral distribution data of an object and that 
of the ambient light with which the object is illuminated. 
The former requires a short processing time and is ad- 
vantageous in terms of cost because the processing is 
simple. On the other hand, the latter requires complex 
processing and therefore is disadvantageous in 
processing time and cost, though it can realize more ac- 
curate color reproduction. 

[0004] Some apparatuses execute simple RGB/spec- 
tral distribution data processing while realizing accurate 
color reproduction by multiple spectral distribution data 
pressing by estimating spectral distribution data on the 
basis of received RGB signals. 
[0005] In each of the above image input apparatuses, 
the internal signal processing system is limited to one 
of the RGB signal processing system, multiple spectral 
distribution data processing system, and RGB/spectral 
distribution data processing system, and has input and 
output units corresponding to one of the processing sys- 
tems. That is, an image input apparatus having an RGB 
signal processing system has an RGB input unit and 
RGB output unit. An image input apparatus having a 
multiple spectral distribution data processing has a 
spectral distribution input unit and spectral distribution 
processing unit. An image input apparatus having an 
RGB/spectral distribution data processing system has 
an RGB input- unit and RGB/spectral distribution 
processing unit. 

[0006] Hence, input and output units for different sig- 
nal processing systems cannot be connected to a single 
image input apparatus. For example, an output unit for 
a spectral distribution data processing system cannot 
be connected to an input unit for an RGB signal process- 
ing system. 



SUMMARY OF THE INVENTION 

[0007] The present invention has been made to solve 
the above-described problems individually or altogeth- 

5 er, and has as a concern to provide image processing 
based on the relationship between the signal processing 
systems of image input and output units. 
[0008] According to a preferred aspect of the present 
invention, the present invention provides an image 

10 processing apparatus comprising: an input unit, ar- 
ranged to input image data; an output unit, arranged to 
output the image data to an external device; and a con- 
troller, arranged to control a signal format of the image 
data to be supplied from the input unit to the output unit 

'5 on the basis of a signal format of image data that can 
be processed by the output unit. 
[0009] In a preferred embodiment, the controller ac- 
quires information related to an input signal format of 
image data that can be input by the input unit and infor- 

20 mation related to an output signal format of the image 
data that can be processed by the output unit and, on 
the basis of the acquired information related to the input 
and output signal formats, controls the signal format of 
the image data to be supplied from the input unit to the 

25 output unit. 

[0010] According to another aspect of the present in- 
vention provides a converter, arranged to convert the 
signal format of the image data input from the input unit, 
wherein when image data having a signal format corre- 

30 spending to the output signal format is not input from the 
input unit, the controller supplies image data obtained 
by converting the signal format by the converter to the 
output unit. 

[001 1 ] Other features and advantages of the present 
35 invention will be apparent from the following description 
taken in conjunction with the accompanying drawings, 
in which like reference characters designate the same 
or similar parts throughout the figures thereof. 

40 BRIEF DESCRIPTION OF THE DRAWINGS 

[0012] 

Fig. 1 is a block diagram showing the functional ar- 
45 rangement of an image input apparatus according 
to the first embodiment; 

Fig. 2 is a block diagram Showing the connection 
state of each unit; 

Fig. 3 is a flow chart showing image input process- 
50 ing; 

Fig. 4 is a block diagram showing the functional ar- 
rangement of an image input apparatus according 
to the second embodiment; 
Fig. 5 is a graph showing examples of color match- 
55 ing functions; 

Fig. 6 is a graph showing an example of an RGB 

spectral distribution characteristic; and 

Fig. 7 is a table showing the correlation between the 
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signal formats of input and output systems and the 
signal formats of image data to be output to the out- 
put system. 

DETAILED DESCRIPTION OF THE PREFERRED 
EMBODIMENTS 

[001 3] Image input apparatuses according to the em- 
bodiments of the present invention will be described be- 
low in detail with reference to the accompanying draw- 
ings. 

First Embodiment 
[Arrangement] 

[001 4] Fig. 1 is a block diagram showing the functional 
arrangement of an image input apparatus according to 
the embodiment. 

[0015] An input unit 201 for photographing an object 
and acquiring image data has a colorimetric data input 
section 202 for acquiring the colorimetric data (RGB da- 
ta in this embodiment) of each pixel and a spectral dis- 
tribution data input section 203 for acquiring the spectral 
distribution data of each pixel. The colorimetric data and 
spectral distribution data acquired in the input unit 201 
are input to an image processing unit 204. 
[0016] The image processing unit 204 is constituted 
by an input system information acquisition section 205 
for acquiring characteristic information of the input sys- 
tem (to be referred to as "input system information" 
hereinafter) such as a signal format in the input unit 201 
(to be referred to as an "input signal format" hereinafter), 
an output system information acquisition section 206 for 
acquiring characteristic information of the output system 
(to be referred to as "output system information" here- 
inafter) such as a signal format in an output unit 208 on 
the output side (to be referred to as an "output signal 
format" hereinafter), and an output signal selection sec- 
tion 207 for selecting the signal format of image data to 
be transferred to the output unit 208 on the basis of the 
acquired input and output system information. The im- 
age processing unit 204 outputs image data having a 
signal format selected by the output signal selection 
section 207. 

[0017] The image data output from the image 
processing unit 204 is subjected to appropriate image 
processing by the output unit 208 and output to an ex- 
ternal device. In the first embodiment, as the output unit 
208, a plurality of processing units using different 
processing signal formats can be used. More specifical- 
ly, one of a colorimetric processing output unit 209 for 
receiving only colorimetric data and processing/output- 
ting the data, a colorimetric/spectral distribution 
processing output unit 21 0 for receiving both colorimet- 
ric data and spectral distribution data and processing/ 
outputting the data, and a spectral distribution process- 
ing output unit 21 1 for receiving only spectral distribution 



data and processing/outputting the data can be con- 
nected to the image processing unit 204 as the output 
unit 208. 

[001 8] Fig. 2 is a block diagram showing the connec- 
5 tion state of each unit of the image input apparatus 
shown in Fig. 1 . 

[0019] Referring to Fig. 2, the input unit 201 has, in 
addition to the above-described colorimetric data input 
section 202 and spectral distribution data input section 

10 203, an image processing section 801 for executing 
processing such as analog-to-digital conversion or 
shading correction for image data having a signal for- 
mat, which is acquired by the colorimetric data input sec- 
tion 202 and spectral distribution data input section 203, 

15 and a communication interface section 802 for transmit- 
ting the processed image data to the image processing 
unit 204 on the output side. 

[0020] The image processing unit 204 has, in addition 
to a selection processing section 805 formed from the 
20 above-described input system information acquisition 
section 205, output system information acquisition sec- 
tion 206, and output signal selection section 207, an im- 
age processing section 804 for executing image 
processing according to the signal format selected by 
25 the selection processing section 805 on the image data, 
and a communication interface section 803 for transmit- 
ting the processed image data to the output unit 208 and 
receiving image data from the input unit 201 . 
[0021] The output unit 208 has a communication in- 
30 terface section 806 for receiving image data from the 
image processing unit 204, an image processing section 
807 for analyzing received image data and generating 
output image data, and an output section 808 for out- 
putting the generated output image data to an external 
35 device (not shown). 

[0022] As shown in Fig. 2, the units (input unit 201 , 
image processing unit 204, and output unit 208) of the 
image input apparatus according to the first embodi- 
ment can exchange image data and the characteristic 
40 information of each unit through the communication in- 
terface sections 802, 803, and 806. 

[Processing] 

45 [0023] Image input processing in the above-de- 
scribed image input apparatus according to the first em- 
bodiment will be described below in detail with reference 
to Fig. 3. 

[0024] Fig. 3 is a flow chart showing processing of 
so causing the input unit 201 to receive colorimetric data 
and spectral distribution data and output image data 
having a signal format that can be processed by the out- 
put unit 208. 

[0025] In step S300, the input unit 201 acquires image 
55 data obtained by photographing an object. In the first 
embodiment, colorimetric data and spectral distribution 
data are acquired by the colorimetric data input section 
202 and spectral distribution data input section 203, re- 
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spectivety. 

[0026] In step S301 , the input system information ac- 
quisition section 205 acquires the input system informa- 
tion of the input unit 201 . In step S302, the output system 
information acquisition section 206 acquires the output 5 
system information of the output unit 208. That is, the 
characteristic information of each of the colorimetric da- 
ta input section 202 and spectral distribution data input 
section 203 in the input unit 201 is acquired as input 
system information, and the characteristic information 10 
of the output unit 208 currently connected to the image 
processing unit 204, i.e., the colorimetric processing 
output unit 209, colorimetric/spectral distribution 
processing output unit 210, or spectral distribution 
processing output unit 21 1 is acquired as output system 15 
information, 

[0027] To cause the output unit 208 connected to the 
image processing unit 204 to process and output the im- 
age data acquired in step S300, processing in steps 
S303 to S307 is executed by the output signal selection 20 
section 207, and image data having a signal format to 
be output to the output unit 208 is selected from the im- 
age data acquired by the input unit 201 . Processing in 
the output signal selection section 207 will be described 
below. 25 
[0028] In step S303, on the basis of the input system 
information acquired in step S301 , pieces of information 
related to the input signal formats of the image data ac- 
quired by the input unit 201 are obtained. The input sig- 
nal formats mean the signal formats of the image data 30 
acquired by the input unit 201 , i.e., a "colorimetric form" 
corresponding to the image data acquired by the color- 
imetric data input section 202 and a "spectral distribu- 
tion form" corresponding to the image data acquired by 
the spectral distribution data input section 203. 35 
[0029] I n step S304, on the basis of the outp ut system 
information acquired in step S302, information related 
to the output signal format of image data that can be 
processed by the output unit 208 is obtained. The output 
signal format means the signal format of image data that 40 
can be processed by the output unit 208 currently con- 
nected to the image processing unit 204, i.e., a "colori- 
metric form" corresponding to the signal format of image 
data that can be processed by the colorimetric process- 
ing output unit 209, the "colorimetric/spectral distribu- 
tion form" corresponding to the signal format of image 
data that can be processed by the colorimetric/spectral 
distribution processing output unit 210, or the "spectral 
distribution form" corresponding to the signal format of 
image data that can be processed by the spectral dis- so 
tribution processing output unit 211 . 
[0030] In step S305, on the basis of the information 
related to the output signal format obtained in step S304, 
the signal format of image data to be output from the 
image processing unit 204 to the output unit 208 is de- ss 
termined. More specifically, when the colorimetric 
processing output unit 209 is connected as the output 
unit 208, the "colorimetric form" is determined. When the 



colorimetric/spectral distribution processing output unit 
210 is connected, both the "colorimetric form" and the 
"spectral distribution form" are determined. When the 
spectral distribution processing output unit 211 is con- 
nected, the "spectral distribution form" is determined. 
[0031] In step S306, image data having the output sig- 
nal format determined in step S305 is selected from the 
image data acquired in step S300. The image data is 
output to the output unit 208 in step S307. 
[0032] The image data output to the output unit 208 
by the above steps has a signal format that matches the 
signal format that can be processed by the output unit 
208. Hence, the output unit 208 can execute predeter- 
mined image processing on the basis of the image data 
received from the Image processing unit 204 to generate 
an appropriate output signal and output the signal to an 
external device. 

[0033] In the first embodiment, in accordance with the 
signal format that can be processed by the output unit 
208 connected to the image processing unit 204, color- 
imetric data is output to the colorimetric processing out- 
put unit 209, both colorimetric data and spectral distri- 
bution data are output to the colorimetric/spectral distri- 
bution processing output unit 21 0, and spectral distribu- 
tion data is output to the spectral distribution processing 
output unit 211. 

[0034] As described above, according to the first em- 
bodiment, image data having a signal format that can 
be processed by the output unit 208 can be selected 
from image data acquired by the input unit 201 and sup- 
plied to the output unit 208. Hence, an image input ap- 
paratus in which the input signal format in the input unit 
201 is not limited to the output signal format in the output 
unit 208 can be implemented. In other words, in an im- 
age input apparatus having the input unit 201 for input- 
ting colorimetric data and spectral distribution data, one 
of the colorimetric processing output unit 209, colorimet- 
ric/spectral distribution processing output unit 210, and 
spectral distribution processing output unit 211 can be 
arbitrarily connected and used as the output unit 208. 
[0035] Generally, as the colorimetric processing out- 
put unit 209, for example, a device for executing simple 
RGB data processing is assumed. However, as the 
colorimetric/spectral distribution processing output unit 
21 0 or spectral distribution processing output unit 211 , 
a multiple spectral distribution data processing device 
for realizing highly accurate color reproduction is as- 
sumed. Hence, according to the first embodiment, in the 
image input apparatus, a highly accurate color repro- 
duction processing device (colorimetric/spectral distri- 
bution processing output unit or spectral distribution 
processing output unit) can be connected while main- 
taining the compatibility with an inexpensive normal de- 
vice (colorimetric processing output unit). 

Second Embodiment 

[0036] The second embodiment of the present inven- 
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tion will be described below. 

[0037] In the above-described first embodiment, an 
arrangement in which the input unit 201 has both the 
colorimetric data input section 202 and the spectral dis- 
tribution data input section 203 has been described. In 
the second embodiment, only a spectral distribution da- 
ta input section 203 is prepared as an input unit, and 
colorimetric data is generated in accordance with the 
output signal format of an output unit 208. 

[Arrangement] 

[0038] Fig. 4 is a block diagram showing the charac- 
teristic functional arrangement of an image input appa- 
ratus according to the second embodiment. The same 
reference numerals as in the above-described first em- 
bodiment denote the same components in Fig. 4, and a 
description thereof will be omitted. 
[0039] The output signal selection section 207 in the 
first embodiment selects image data having a signal for- 
mat to be output from image data having signal formats 
acquired by the input unit 201 . However, an output sig- 
nal selection section 404 in the second embodiment de- 
termines the signal format of image data to be output on 
the basis of the output signal format of the output unit 
208 independently of the input signal format of an input 
unit 201. 

[0040] As in the first embodiment, an output system 
information acquisition section 406 in an image process- 
ing unit 402 shown in Fig. 4 acquires the output system 
information of the output unit 208 (colorimetric process- 
ing output unit 209, colorimetric/spectral distribution 
processing output unit 210, or spectral distribution 
processing output unit 211) currently connected to the 
image processing unit 402. In the second embodiment, 
since only the spectral distribution data input section 
203 is prepared as the input unit, no input system infor- 
mation acquisition section for acquiring input system in- 
formation is prepared. However, an input system infor- 
mation acquisition section may be prepared, as in the 
first embodiment. 

[0041 ] On the basis of the output system information 
acquired by the output system information acquisition 
section 406, the output signal selection section 404 de- 
termines the signal format of image data to be output 
from the image processing unit 402 to the output unit 
208 from signal formats that are prepared in advance. 
The signal formats that are prepared in advance are the 
"colorimetric form", "colorimetric/spectral distribution 
form", and "spectral distribution form". 
[0042] In the second embodiment, since only the 
spectral distribution data input section 203 is prepared 
as the input unit, no colorimetric data is input. Hence, 
when the "colorimetric form" or "colorimetric/spectral 
distribution form" is selected by the output signal selec- 
tion section 404, a colorimetric data generation section 
403 generates colorimetric data on the basis of spectral 
distribution data input from the spectral distribution data 



input section 203. The colorimetric data generation 
method will be described later in detail. 
[0043] Upon receiving the spectral distribution data 
from the spectral distribution data input section 203 and 

5 the colorimetric data from the colorimetric data genera- 
tion section 403, an output signal switching section 405 
switches image data to be output from the image 
processing unit 402 on the basis of the output signal for- 
mat determined by the output signal selection section 

10 404. More specifically, when the output signal format is 
the "colorimetric form", the output signal switching sec- 
tion 405 outputs only the colorimetric data. When the 
output signal format is the "colorimetric/spectral distri- 
bution form", the output signal switching section 405 

15 outputs both the colorimetric data and the spectral dis- 
tribution data. When the output signal format is the 
"spectral distribution form", the output signal switching 
section 405 outputs only the spectral distribution data. 



[0044] The colorimetric data generation method by 
the colorimetric data generation section 403 will be de- 
scribed here. 

25 [0045] As the first method, received spectral distribu- 
tion data is temporarily converted into device-independ- 
ent colorimetric vector data and then converted into 
colorimetric data on the target colorimetric system (e.g., 
RGB system). 

30 [0046] That is, when spectral distribution data re- 
ceived from the spectral distribution data input section 
203 is integrated by convolution calculation of a color 
matching function in an appropriately selected colori- 
metric system, the spectral distribution data can be con- 

35 verted into colorimetric vector data (tristimulus values) 
having ternary values. As colorimetric vector data hav- 
ing ternary values, a device-independent colorimetric 
system such as an L*a*b # colorimetric system or XYZ 
colorimetric system is generally employed. Fig. 5 is a 

40 graph showing examples of color matching functions in 
the XYZ colorimetric system. 

[0047] As the second method, three data are selected 
from a plurality of wavelength data contained in the re- 
ceived spectral distribution data, and these data are 

45 converted into ternary values of colorimetric data. 
[0048] That is, the spectral characteristic of target 
colorimetric data is held in advance. Only data corre- 
sponding to the spectral characteristic of the colorimet- 
ric data are extracted from received spectral distribution 

50 data and converted into colorimetric data. For example, 
assume that colorimetric data contains RGB signals. 
Since RGB signals have spectral characteristics shown 
in Fig. 6, three wavelength data that exhibit the peaks 
of the RGB colors are extracted from the received spec- 

55 tral distribution data. On the basis of these data, RGB 
signal values can be generated. 
[0049] As described above, according to the second 
embodiment, image data having an unacquired signal 
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format is generated as needed, thereby outputting im- 
age data having a signal format that can be processed 
by the output unit 208. Hence, as in the first embodi- 
ment, in an image input apparatus for acquiring only 
spectral distribution data, one of the colorimetric 
processing output unit 209, colorimetric/spectrat distri- 
bution processing output unit 21 0, and spectral distribu- 
tion processing output unit 211 can be arbitrarily con- 
nected and used as the output unit 208. 

Other Embodiment 

[0050] In the first embodiment, a case wherein the in- 
put unit 201 acquires colorimetric data and spectral dis- 
tribution data has been described. In the second em- 
bodiment, a case wherein only spectral distribution data 
is acquired has been described. However, the present 
invention is not limited to these examples. Fig. 7 shows 
the correlation between the input signal format, the out- 
put signal format, and the signal format of image data 
to be output to the output unit. Referring to Fig. 7, the 
column wherein the input signal format is "colorimetric 
and spectral distribution form" corresponds to the 
above-described first embodiment, and the column 
wherein the input signal format is "spectral distribution 
form" corresponds to the second embodiment. 
[0051] According to Fig. 7, when the input signal for- 
mat is the "colorimetric form", i.e., when only colorimet- 
ric data is input from the input unit, no spectral distribu- 
tion data can be formed on the basis of the colorimetric 
data. Hence, only when the output signal format is the 
"colorimetric form", appropriate image data (colorimetric 
data) can be output to the output unit. Otherwise, i.e., 
when the output signal format is the "colorimetric/spec- 
tral distribution form" or "spectral distribution form", im- 
age data having an appropriate signal format cannot be 
output to the output unit. 

[0052] The correlation table shown in Fig. 7 is held in 
a memory (not shown) in the image input apparatus and 
looked up in, e.g., determining the output signal format 
on the basis of the signal formats of the input and output 
units in step S305 of Fig. 3. If the input signal format is 
the "colorimetricform", as described above, and the out- 
put signal has a form other than the "colorimetric form", 
i.e . , when image data cannot be output to the output unit, 
the user is preferably notified of the error in step S305. 
[0053] As described above, for all combinations of in- 
put and output signal formats, the signal format of cor- 
responding image data is determined in advance. With 
this arrangement, not only as the output unit but also as 
the input unit, an input unit for receiving only colorimetric 
data, an input unit for receiving both colorimetric data 
and spectral distribution data, or an input unit for receiv- 
ing only spectral distribution data can be arbitrarily con- 
nected to the image input apparatus and used. 
[0054] Hence, the input and output units can be de- 
tached from the image input unit. Even when the at- 
tached input and output units use different signal for- 



mats, an appropriate image processing can be execut- 
ed. 

[0055] In the above-described embodiments, RGB 
data is most generally used as colorimetric data. 

5 [0056] As many apparently widely different embodi- 
ments of the present invention can be made without de- 
parting from the spirit and scope thereof, it is to be un- 
derstood that the invention is not limited to the specific 
embodiments thereof except as defined in the append- 

10 ed claims. 



Claims 

15 1 . An image processing apparatus comprising: 



an input unit, arranged to input image data; 
an output unit, arranged to output the image da- 
ta to an external device; and 
20 a controller, arranged to control a signal format 

of the image data to be supplied from said input 
unit to said output unit on the basis of a signal 
format of image data that can be processed by 
said output unit. 

25 

2. The apparatus according to claim 1 , wherein said 
controller acquires information related to an input 
signal format of image data that can be input by said 
input unit and information related to an output signal 

30 format of the image data that can be processed by 
said output unit and, on the basis of the acquired 
information related to the input and output signal 
formats, controls the signal format of the image data 
to be supplied from said input unit to said output 

35 unit. 

3. The apparatus according to claim 2, further com- 
prising a converter, arranged to convert the signal 
format of the image data input from said input unit, 

4Q wherein when image data having a signal for- 

mat corresponding to the output signal format is not 
input from said input unit, said controller supplies 
image data obtained by converting the signal format 
by said converter to said output unit. 

45 

4. The apparatus according to claim 1, wherein said 
input unit comprises an image sensing device for 
acquiring image data of an object. 

so 5. The apparatus according to claim 1 , wherein the 
signal format includes a colorimetric form, spectral 
distribution form, and colorimetric and spectral dis- 
tribution form. 

55 6. The apparatus according to claim 5, wherein image 
data having the colorimetric form is RGB data. 

7. The apparatus according to claim 1 , wherein said 
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input unit and/or output unit can be detached. 

8. A control method of an image processing apparatus 
which has an input unit arranged to input image data 
and an output unit arranged to output the image da- s 
ta to an external device, comprising the step of: 

controlling a signal format of the image data to 
be supplied from the input unit to the output unit 
on the basis of a signal format of the output unit. 10 

9. The method according to claim 8, wherein in said 
controlling step, information related to an input sig- 
nal format of image data that can be input by the 
input unit and information related to an output signal 15 
format of the image data that can be processed by 
the output unit are acquired, and on the basis of the 
acquired information related to the input and output 
signal formats, the signal format of the image data 

to be supplied from the input unit to the output unit 20 
is controlled. 

10. The method according to claim 9, further compris- 
ing the steps of: 

25 

converting the signal format of the image data 
input from the input unit; and 
when image data having a signal format corre- 
sponding to the output signal format is not input 
from the input unit, supplying image data ob- 30 
tained by converting the signal format to the 
output unit. 

11. A computer program product storing a computer 
readable medium comprising a computer program 35 
code, for a control method of an image processing 
apparatus which has an input unit arranged to input 
image data and an output unit arranged to output 
the image data to an external device, comprising 
process procedure code for controlling a signal for- 40 
mat of the image data to be supplied from the input 
unit to the output unit on the basis of a signal format 

of the output unit. 

12. The product according to claim 11 , wherein in said 45 
controlling process, information related to an input 
signal format of image data that can be input by the 
input unit and information related to an output signal 
format of the image data that can be processed by 
the output unit are acquired, and on the basis of the so 
acquired information related to the input and output 
signal formats, the signal format of the image data 

to be supplied from the input unit to the output unit 
is controlled. 

55 

13. The product according to claim 12, further compris- 
ing process procedure code for: 



converting the signal format of the image data 
input from the input unit; and 
when image data having a signal format corre- 
sponding to the output signal format is not input 
from the input unit, supplying image data ob- 
tained by converting the signal format to the 
output unit. 
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